Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix invalid regex (fixes #159) #163

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

slevithan
Copy link

@slevithan slevithan commented Dec 17, 2024

See extended details at #159, but essentially, \xHH for values above 7F doesn't work the same in Oniguruma (the regex engine used for TextMate grammars) as in other flavors of regex. As a result, a standalone unenclosed \xff like the one corrected here is an invalid UTF-8 encoded byte value, rather than a valid code point value as it would be if using the enclosed version \x{ff}.

This error is currently leading to highlighting bugs with some input strings (code points above FF are not matched by this negated range), and preventing the Hack grammar from working at all with Shiki's JS engine.

The adjacent \x7f that I also changed was already valid, but I changed it for consistency.

Note that this was the only place the invalid \\x7f-\\xff appeared, but the valid/correct version \\x{7f}-\\x{ff} already appears 28 times in the same file.

@slevithan
Copy link
Author

@PranayAgarwal, is anything else needed for this to land?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant